Regularization methods allow one to handle a variety of inferential problemswhere there are more covariates than cases. This allows one to consider apotentially enormous number of covariates for a problem. We exploit the powerof these techniques, supersaturating models by augmenting the "natural"covariates in the problem with an additional indicator for each case in thedata set. We attach a penalty term for these case-specific indicators which isdesigned to produce a desired effect. For regression methods with squared errorloss, an $\ell_1$ penalty produces a regression which is robust to outliers andhigh leverage cases; for quantile regression methods, an $\ell_2$ penaltydecreases the variance of the fit enough to overcome an increase in bias. Theparadigm thus allows us to robustify procedures which lack robustness and toincrease the efficiency of procedures which are robust. We provide a generalframework for the inclusion of case-specific parameters in regularizationproblems, describing the impact on the effective loss for a variety ofregression and classification problems. We outline a computational strategy bywhich existing software can be modified to solve the augmented regularizationproblem, providing conditions under which such modification will converge tothe optimum solution. We illustrate the benefits of including case-specificparameters in the context of mean regression and quantile regression throughanalysis of NHANES and linguistic data sets.
展开▼
